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Abstract 

We consider the problem of recovering a signal consisting of a superposition of point sources 
from low-resolution data with a cut-off frequency fc- If the distance between the sources is under 
l//c, this problem is not well posed in the sense that the low-pass data corresponding to two 
different signals may be practically the same. We show that minimizing a continuous version of 
the £i norm achieves exact recovery as long as the sources are separated by at least 1.26//c. The 
proof is based on the construction of a dual certificate for the optimization problem, which can 
be used to establish that the procedure is stable to noise. Finally, we illustrate the flexibility of 
our optimization-based framework by describing extensions to the demixing of sines and spikes 
and to the estimation of point sources that share a common support. 

Keywords. Super-resolution, line-spectra estimation, convex optimization, dual certificates, sparse 
recovery, overcomplete dictionaries, group sparsity, multiple measurements. 


1 Introduction 


Extracting fine-scale information from low-resolution data is a major challenge in many areas of the 
applied sciences. In microscopy, astronomy and any other application employing an optical device, 
spatial resolution is fundamentally limited by diffraction [^. Figure [^illustrates a popular model 
for the data-acquisition process in such cases: the object of interest is convolved with a point-spread 
function that blurs the hne-scale details, acting essentially as a low-pass filter. The problem of super¬ 
resolution is that of reconstructing the original image from the blurred measurements. An analogous 
challenge often arises in signal-processing: estimating the spectrum of a signal from a finite number 
of samples. Truncating the signal in the time domain limits the spectral resolution, as shown in 
the lower half of Figure Spectral super-resolution, or equivalently line-spectra estimation, is the 
problem of recovering the spectrum of the original signal from the truncated data. 


By super-resolution we mean the inverse problem of estimating a signal from low-resolution mea¬ 
surements, but the term may have other meanings in different contexts. In optics, it often refers to 


the problem of overcoming the diffraction limit by modifying the data-acquisition mechanism 50 
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Figure 1: Schematic illustration of spatial and spectral super-resolution. 
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In image-processing, it denotes the problem of upsampling an image onto a finer grid while pre¬ 
serving its edge structure and hallucinating high-frequency textures in a reasonable way |52| . To be 
clear, in this work we focus on recovering the lost fine-scale features without altering the low-pass 
sensing process. 


In order to super-resolve a signal it is necessary to leverage some prior knowledge about its structure. 
Otherwise the problem is hopelessly ill posed, since the missing spectrum can be filled in arbitrarily 
to produce estimates that correspond to the data. Here, we consider signals that may be represented 
as superpositions of point sources, such as celestial bodies in astronomy 41 , neuron spikes in 
neuroscience 56 or line spectra in signal processing and spectroscopy 45,66 . In addition, locating 


pointwise fluorescent probes is a crucial step in some optical super-resolution procedures capable of 
handling more complicated objects, such as photoactivated localization microscopy (PALM) [^[44] 
or stochastic optical reconstruction microscopy (STORM) [60]. 


At an abstract level, the deconvolution of point sources or spikes from bandlimited data is an in¬ 
stance of a central question in modern data processing: how to recover a low-dimensional object 
embedded in a high-dimensional space from incomplete linear measurements. Nonparametric tech¬ 
niques based on convex optimization have had great success in tackling problems of this flavor. 
Notable examples include sparse regression in high-dimensional settings 
and matrix completion 


74 , compressed sens¬ 


ing 24,32 


21 . The interest of developing optimization-based methods 


for super-resolution lies in their robustness to noise and in their flexibility to account for differ¬ 
ent structural assumptions on the signal, noise and measurement model. In recent work, convex 
programming has been shown to recover a superposition of point sources exactly from bandlimited 
data, as long as the sources are separated by a minimum distance of 2//c, where fc is the cut-off 
frequency of the sensing process 18 . Subsequent publications have established 


that the method is robust to noise in non-asymptotic regimes. 


The goal of the present paper is to further develop this line of research through two main contri¬ 
butions: 


• We establish that it is possible to super-resolve signals with minimum separations above 
1.26//c via convex programming. Section provides the context for this result by describing 
the basic super-resolution problem and our optimization-based approach. The proof, which 
is based on the construction of a novel dual certificate that also allows to extend previous 
stability results, is presented in Section 

• In Sectionj^we illustrate the flexibility of our approach by adapting it to two related problems: 
demixing of sines and spikes and super-resolution of multiple signals that share a common 
support. In each case we propose a optimization program tailored to the problem, discuss 
how to solve it, analyze its optimality conditions and provide some numerical simulations. 
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2 Super-resolution of point sources 

2.1 Basic model 


We model a superposition of point sources as a sum of weighted Dirac measures supported on a 
subset T of the unit interval 

( 2 . 1 ) 

tj&T 

where 5r is a Dirac measure at r and the amplitudes aj may be complex valued. We study the 
problem of estimating such a signal from low-resolution measurements which correspond to the 
convolution between the signal and a low-pass point spread function (PSF) (j), 


a^LR {t) := (p*x{t) = ^ aj(j) {t - tj) , (2.2) 

tj&T 

as illustrated at the top of Figure [TJ If the cut-off frequency of the PSF is equal to fc, in the 
frequency domain the measurements are of the form 

.Fxlr = J^cj) J='x = '$ 'n.i-f.j,] Tx, 

where J- f denotes the Fourier transform of a function or measure / and is an indicator 

function that is zero out of the interval [—fc, fc]- For ease of exposition, we assume that the Fourier 
transform of the PSF is constant over [—fc, fc], he- the PSF is a periodized sine or Dirichlet kernel, 
but our results hold for any PSF with a known low-pass spectrum. Since the support of xlr is 
restricted to the unit interval, it follows from the sampling theorem that its spectrum is completely 
determined by the discrete samples 

y{k) = .FxLR(fc) = [' e-“x(dt) = k e Z, \k\ < fc, (2.3) 

Jo ^ 


where we assume for simplicity that fc is an integer. In a more compact form, the sensing process 
can be represented as 

y = FnX (2.4) 

where y G C"' and Tn is the linear operator that maps a measure or function to its lowest n := 2/c-|-l 
Fourier coefficients. 


If the signal x is used to model a superposition of line spectra, equation (2.4) has a very natural 
interpretation: the data y correspond to a finite number of samples of the signal in the time domain. 
As sketched in the lower half of Figure truncating the signal in the time domain is equivalent 
to convolving its spectrum with a sine function. Our model can consequently be applied directly 
to spectral super-resolution, where the aim is to estimate sparse line spectra from time-domain 
samples. 


2.2 Minimum separation 

In contrast to compressed sensing, where randomized measurements preserve the energy of ar¬ 
bitrary sparse signals with high probability (this is commonly known as the restricted isometry 
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property [23]), sparsity is not a strong enough prior to ensure that the super-resolution problem is 
well posed. Indeed, low-pass filtering may suppress sparse signals almost entirely if their support 
is too clustered together. As a result, in order to derive meaningful guarantees for super-resolution 
it is necessary to impose conditions on the signal support. To this end, we define the minimum 
separation of the support of a signal, as introduced in . 

Definition 2.1 (Minimum separation). Let T he the cirele obtained by identifying the endpoints 
on [0,1]. For a family of points T C T, the minimum separation (or minimum distance) is defined 
as the closest distance between any two elements from T, 

A(T)= inf \t-t'\. (2.5) 

where \t — t'\ is the distance (maximum deviation in any coordinate). To he clear, this is the 
wrap-around distanee so that the distanee between t = 0 and t' = 3/4 is equal to 1/4. 


If the minimum distance is too small with respect to the cut-off frequency of the data, it may 
become impossible to estimate the signal even under very small perturbations to the data. A 
fundamental limit in this sense is Ac := l//c, the inverse of the cut-off frequency, which also 
corresponds to the width of the main lobe of the point-spread function (j). The reason is that for 
minimum separations just below Ac/2 there exist signals that lie almost in the null space of the 
low-pass operator defined by (2.4 If such a signal d corresponds to the difference between two 


different signals si and S 2 so that si —S 2 = d, it will be very challenging to distinguish si and S 2 from 
low-resolution data. Figure [^illustrates this: the measurements corresponding to two signals with 
disjoint supports and a minimum distance of 0.9Ac for /c = 10^ are indeed almost indistinguishable. 
The phenomenon can be characterized theoretically in an asymptotic setting using Slepian’s prolate- 


spheroidal sequences (see also Section 3.2 in |18j). More recently, Theorem 1.3 of 53 provides 
a non-asymptotic analysis. Finally, other works have obtained lower bounds on the minimum 


separation necessary for convex-programming approaches to succeed [36 ,70 


2.3 Optimization-based super-resolution 


Our approach to super-resolution is based on optimization: we estimate the signal by minimizing a 
sparsity-inducing norm. Since we are interested in point sources that may be supported at arbitrary 
locations within a continuous interval, we resort to a continuous counterpart of the ii norm known 
as the total-variation norn:0 If we consider the space of measures supported on the unit interval, 
this norm is dual to the infinity norm (see Section A in the appendix of 18 for a different definition), 
so that for a measure x we have 


l®l Itv ~ 


sup Re 
,<i./eC(T) 


/ it)x (dt) 


is the notorious Rayleigh resolution limit 31 , below which it is challenging to even distinguish two neigh¬ 
boring sources. 

^The term total variation may also refer to the norm of the discontinuities of a piecewise-constant function, 


which is a popular regularizer in image processing and other applications 59 . 
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Spectrum (magnitude) 




Figure 2: Two signals with disjoint supports that satisfy the minimum-separation condition 
for A (T) = 0.9fc when fc = 10^ (top left) and their spectrum (top right). Their difference 
(center left) has a spectrum that is concentrated away for the low-pass band between —/^ and 
fc (center right). As a result, it is very difficult to the data corresponding to the two signals 
(bottom). 
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Figure 3: Graphs showing the fraction of times Problem (2.6 1 achieves exact recovery over 
10 trials with random signs and supports. A phase transition occurs near Ac. The simulations 
are carried out using the high-precision semidefinite-programming solver CVX [42] . 


For a superposition of Dirac deltas the total-variation norm is equal to the ii norm of 

the coefficients, i.e. Ha^H-pv ~ super-resolution method consists in minimizing the 

total-variation norm of the estimate subject to data constraints, as proposed in (T^, 


mm X 


TV 


subject to J^n X = y, 


( 2 . 6 ) 


where the minimization is carried out over the set of all finite complex measures x supported on 


[0,1]. Section 2.7 discusses how to solve this optimization problem. 


In it was established that if the minimum separation of the support of a signal A(T) is greater 
or equal to 2Ac, TV-norm minimization achieves exact recovery. However, numerical experiments 
indicate that the actual limit at which super-resolution via TV-norm regularization may fail is Ac- 
This is shown in Figure]^ see also Section 5 of 18 . As explained in Section 2.2 Ac is a natural 
limit in the sense that below that minimum separation the problem may become ill posed. Our 
main result is that the guarantee for exact recovery can be extended to a minimum separation of 
just 1.26Ac. 

Theorem 2.2. Let T = {tj} be the support of x. If the minimum separation obeys 

A{T) > 1.26//c = 1.26Ac, 

then X is the unique solution to (2.6). This holds as long as fc > 10^. 


(2.7) 


Since the signal is assumed to be supported on the unit interval, the result implies that it is possible 
to recover a number of point sources that is directly proportional to the cut-off frequency. 

2.4 Dual certificate 


Theorem 2.2 is a direct consequence of Proposition |2.3| below, which establishes the existence of 


a certain subgradient of the TV norm that is orthogonal to the null space of the measurement 
operator. Such an object is often referred to as a dual certificate in the compressed-sensing liter¬ 
ature |22| because it certifies that exact recovery occurs and its coefficients are a solution to the 


dual of Problem (2.6). 
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Proposition 2.3. Under the conditions of Theorem 2.2 for any sign pattern v G sueh that 

\vj\ = 1 for all j, there exists a low-pass trigonometric polynomial 


q{t) = Y, 

k=-fc 

obeying 

q{tj)=vj, tj€T, 

|g(t)| < 1, tj i T, 

In addition, there exist numerical constants Cq G (0,1), Ci and C 2 such that 


1 - Ci/2 (t - tjf < |g(t)| < 1 - C 2/2 {t - tjf , 


\t — tj\ < CqXc, tj G T. 


( 2 . 8 ) 

(2.9) 

( 2 . 10 ) 

( 2 . 11 ) 


By (2.9) and (2.10), the polynomial is a subgradient of the total-variation norm at the original 


signal X. By (2.8) it is also low pass, which means that it is orthogonal to the null space of the 


measurement operator, as any signal in the null space is high pass. This immediately implies that 
for any signal h in the null space 


\x + 


Itv — 


Itv + = 


Itv ■ 


The bound on the off-support ( |2.10 ) actually implies that this inequality is strict, so that x is 
the unique solution to Problem |2.6[ A complete proof of the fact that Theorem 2.2 follows from 


Proposition |2.8| is provided in Section A of the appendix o f [18| . The quadratic bound (2.11) is key 
in establishing robustness guarantees, see (19,39 . Section |4| is devoted to proving Proposition 2.3 


In the case of super-resolution, the dual certihcate is a trigonometric polynomial with cut-off fre¬ 
quency fc that interpolates the sign of the original signal on its support and has magnitude strictly 
bounded by one on the off-support (see Proposition 2.3). In (^, such a dual certificate is con¬ 


structed using interpolation with a low-pass kernel and its derivative. The proof of Proposition 2.3 
generalizes this approach, allowing to optimize the choice of the interpolation kernel by using sharp 
non-asymptotic bounds on the Dirichlet kernel; we defer the details to Section 

Finally, we would like to emphasize that the structure of the dual polynomial reveals what signals 
will be more challenging for the optimization-based procedure. If the support of the signal is 
cluttered together and the sign of its coefficients varies rapidly, it may not be possible to achieve 
the interpolation with a bounded low-pass polynomial. In Sections 3.1.3 and 3.2.3 we show that 


similar insights arise when we derive dual certificates for other optimization programs designed to 
tackle extensions of the basic super-resolution problem. 

2.5 Robustness to noise 

In any problem involving real data, it is necessary to account for perturbations and model imperfec¬ 


tions. In the case of super-resolution, we can adapt Problem (2.6) by using an inequality constraint 
to quantify the uncertainty. 


mm X 


TV 


subject to \\FnX — y ||2 < 


( 2 . 12 ) 






















where 6 is an estimate of the noise level. Alternatively, we could also consider a Lagrangian 
formulation of the form 


mm X 


\ty + ^ W^nX - y\\l, 


(2.13) 


where the regularization parameter 7 > 0 governs the tradeoff between data fidelity and the sparsity 
of the estimate. 

Recent works [3l [T^[Ml[7l] derive non-asymptotic guarantees on the estimation error achieved when 
solving these problems to perform super-resolution from noisy data. The proofs of these stability 
guarantees rely in part on the dual certificate constructed in 18 and on generalizations of this 


construction. As a result, the guarantees only hold under the proviso that the minimum separation 
is greater or equal to 2 Ac. 

The techniques developed to prove Theorem |2.2[ which are presented in Section allow to extend 
these results to minimum separations of just 1.26Ac. In more detail. Lemma 2.7 in constructs a 
low-pass polynomial q such that q {tj) = 0 and | q' {tj ) | = 1 for all tj belonging to the support of the 
original signal (i.e. the polynomial is locally linear). The polynomial is built through interpolation, 
using the same low-pass kernel as in 18 . The construction can consequently be adapted by using 


the interpolation kernel described in Section along with the bounds provided in Section 4.1 
The same holds for the results in In this case, a low-pass polynomial that is equal to one 


on a certain element of the support and to zero on the rest is used to obtain support-detection 
guarantees. The polynomial is constructed in Lemma 2.2 of 39 , again using the same interpolation 
kernel as in [^. The kernel and bounds described in the present work can be leveraged to build 
such a polynomial for supports with minimum separations above 1.26Ac. Finally, our techniques 


can also be used to sharpen the analysis in 54 , which studies the super-resolution of point sources 
with positive amplitudes. 


2.6 Extensions 


Apart from the extensions discussed in Section there are several interesting extensions to the 
basic super-resolution problem. In compressed sensing off the grid the aim is to super-resolve a 
sparse signal from a random subset of its low-pass Fourier coefficients. Exact recovery via convex 
programming has been shown to occur with high probability for a number of measurements that is 
proportional to the sparsity level (up to logarithmic factors) as long as the support of the signal has 
a minimum separation of 2Ac [^. This bound can be lowered to 1.26Ac using the dual-certificate 
construction in Section]^ Our construction also allows to sharpen results pertaining to the recovery 
of non-uniform splines from their projection onto spaces of algebraic polynomials [^[^. Finally, 
it is straightforward to apply our results to the recovery of piecewise-constant or piecewise-smooth 


functions from low-pass data. The details can be found in Section 1.10 of 18 . 


2.7 Implementation 


Solving Problem (2.6) may seem challenging at first because its primal variable is infinite dimen¬ 
sional. This may be overcome by discretizing the unit interval into a grid and solving an £i-norm 
minimization problem. However, it is also possible to solve the problem exactly without resorting 
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to discretization. A strategy to achieve this is to recast the dual problem as a tractable semidefi- 
nite program and then decode the support of the primal variable from the dual solution. The dual 
problem is 


max (y, c) subject to 

cSC" 




(2.14) 


where the inner product is defined as {y,c) := Re {y*c). The dual variable c is hnite-dimensional but 
the constraint is infinite dimensional; the magnitude of the trigonometric polynomial (c) (t) = 
Yl\k\<fc must be bounded by 1. Fortunately, this condition has a semidefinite representation 

provided by the following proposition, which is a consequence of the Fejer-Riesz Theorem (see 
Section A.l of the appendix for the proof and Theorem 4.24 in 35 for a more general result). 

Proposition 2.4. Let C G [gt Ck denote the kth column of C, 

m 

Y,\{KCk){t)\^ <1 for all ter 

k=l 

if and only if there exists a Hermitian matrix A G obeying 

hO, T*{A) = ei, (2.15) 

where ei is the first vector of the canonical basis o/M"'. 


A C 
C* I 


For any vector u such that ui is positive and real, T (u) is a Hermitian Toeplitz matrix whose first 
row is equal to u. The adjoint of T with respect to the usual matrix inner product {Mi, M 2 ) = 
Tr (MlM 2 ), extracts the sums of the diagonal and off-diagonal elements of a matrix 

n-j+l 

T*{M)^= ^ 

i=l 


Setting m = 1 in Proposition 2.4| Problem (|2.14|) is equivalent to 

^ 0, T* (A) = ei. 


max {y, c) 
ceC", AeC"><" 


subject to 


A c 
c* 1 


(2.16) 


In order to extract an estimate of the support of the primal variable, we construct a support-locating 
polynomial from the solution to (2.16) c. 


Pa{t) = {Kc)it). 


(2.17) 


By strong duality, which holds because the interior of the feasible set of Problem (2.14) contains 
the origin and is consequently non-empty 57 , any solution x to (2.6) obeys 

/■i 


Re 


Pc {t) x{dt) 
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= {^nPx) = {c,Pnx) = {y,c) = P||tv 


which implies that Pc is equal to the sign of the primal solution x at any point where the latter is 
non-zero. This suggests super-resolving the signal by applying the following scheme: 
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Figure 4: On the left we have the low-pass measurements (blue) corresponding to a super¬ 
position of spikes (red). At the center, we see how the support-locating polynomial obtained 
from solving Problem (2.16) interpolates the sign of the signal very precisely. This allows to 
obtain an extremely accurate estimate of the signal, as shown on the right. 


1. Solve the finite-dimensional semidefinite program (2.16). 


2 . Construct the support-locating polynomial and determine a set of points where its magnitude 
is equal to one to produce an estimate of the signal support. 

3. Estimate the amplitude of the signal by solving the corresponding system of equations. 


18 and Section 3 


Eigure shows an example. This approach was proposed in 
in 1^. We refer the reader to Section 4 in 
and a more detailed discussion (see also 
denoising of line spectra) 


and extended to a noisy setting 
for numerical simulations 


m 


19 


12 where a related semidefinite program is applied to the 


Sections 3.1.2 and 3.2.2 describe similar algorithms for demixing sines 

Finally, we would like to 


and spikes and super-resolving point sources with a common support 
mention that other strategies to overcome discretization have been proposed 15l 16 . 


2.8 Related work 


A popular approach to the estimation of point sources from bandlimited data is to fit an estimate of 
the point-spread function of the sensing device to each source individually; see [44| for an example 
in fluorescence microscopy. This is essentially equivalent to matched-filtering techniques used in 
communications and radar, as well as linear nonparametric spectral-analysis algorithms such as the 


periodogram 68 . These simple methods perform well as long as the dynamic range of the signal is 
not very high and the sources are well separated. However, if this is not the case, their performance 
can be degraded due to aliasing, even in the absence of noise. 

Prony’s method is a classical technique that exploits the algebraic structure of the data to tackle 
the problem of spectral super-resolution |55|. The method is able to super-resolve the support of 


the sparse spectrum of a signal from measurements following (2.3) by building a polynomial whose 
zeros are located exactly on the support. This motivates the use of the term annihilating filter 
in the finite rate of innovation (FRI) framework [M ,34,77 . The FRI framework allows to super¬ 


resolve signals with a parametric representation that has a finite number of degrees of freedom. 
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The superpositions of Dirac measures that we consider in spectral super-resolution belong to this 
For this type of signal, FRI reduces exactly to Prony’s method (see Section 3 in 


77 


or 


class. 

Section 1 in |14|). We would like to mention that algebraic techniques have also been applied to 
the location of singularities in the reconstruction of piecewise polynomial functions from a finite 
number of Fourier coefficients (see 37 and references therein). The theoretical analysis of these 


methods proves their accuracy up to a certain limit related to the number of measurements. 

In the absence of noise, Prony’s method is capable of super-resolving s spikes from 2s measurements 
without any minimum-separation condition. However this is no longer the case when the data are 
noisy, as small perturbations may produce substantial changes in the position of the zeros of the 
annihilating polynomial. A popular approach to overcome this issue is to estimate the location 
of the sources from the eigendecomposition of the empirical covariance matrix of the data. Such 
techniques, which include MUSIC 13,64 and ESPRIT 
signal-processing community. We refer the reader to |68l 


58 , have widespread popularity in the 


for a detailed description. A variant that 
is often used by FRI methods fits a low-rank Toeplitz model to a matrix constructed from the 
data, see Section 2 of [14] . This technique is due to Tufts and Kumaresan and is also known 
as Cadzow denoising [ITP 

Most theoretical work analyzing the robustness of covariance-based methods for spectral super¬ 
resolution is based on the asymptotic characterization of the sample covariance matrix under 
Gaussian noise 27,69 . However, some recent stability analysis has shown that these methods 
are robust for signals with a minimum separation above Ac 53 , see also p0,49 . In addition, |49| 


shows that numerically the noise level tolerated by MUSIC for small signal separations obeys a 
power law. In practice, the performance of classical spectral methods such as MUSIC, Cadzow’s 
method or ESPRIT seems to be comparable to the optimization-based approach which is the focus 
of the present paper, see [12] for some numerical comparisons. 


The idea of applying £i-norm minimization and related optimization programs to the super- 


resolution problem dates back to Beurling [10,11 and researchers in geophysics from the 1970s 


4(1 


46|[^ . Theoretical guarantees for positive signals in the absence of noise were obtained in 28|[33 
(see also [O for an interesting recent result). As explained previously in this section, 18 proved 


that exact recovery via convex optimization for measures with arbitrary complex amplitudes sat¬ 
isfying a minimum separation of 2Ac and provided stability guarantees for a discretized version 
of the problem. Subsequent works then established stability guarantees in a continuous set¬ 
ting [3 , 19,36 39,71 and analyzed related problems using similar techniques il [ 43 ]. Finally, 


we would like to mention an application of super-resolution via convex programming to fluores¬ 
cence microscopy [SO] , another optimization-based approach to super-resolve point sources on a 
continuous domain 38 and an alternative proof of exact recovery via TV-norm minimization [67[ 
that allows to sharpen the result in [T^ to 1.56Ac. 


3 A general framework for super-resolution 

In practical applications it is often desirable to adapt the signal, noise or measurement model 
to incorporate prior information. Optimization-based methods are well suited for this purpose: 
tailoring the cost function and the constraints of the optimization program allows to account 
for application-specific structure. We illustrate this by deriving optimization-based algorithms 
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Figure 5: Mixture of sines and spikes: samples from a multi-sinusoidal signal (top) are 
combined with impulsive events (middle) to yield the data (bottom). 


to demix sines and spikes and to super-resolve multiple signals with a common support. In both 
cases we provide an implementation based on semidehnite programming, derive the corresponding 
dual certificate and report some experimental results. 


3.1 Super-resolution from corrupted data: Demixing of sines and spikes 


As mentioned previously, the super-resolution problem discnssed in Section |2.1| can be used to 
model line-spectra estimation. In that case, the measurements correspond to equispaced samples 
from a mnlti-sinnsoidal signal measnred at the Nyquist rate over a limited period of time. We now 
consider line-spectra estimation when some of the data may be completely corrupted. This could 
be due to the presence of sparse noise, to the intermittent failure of a sensor or to some other 
impnlsive phenomenon. We model the corrnptions as a vector s G with support S. In the 
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resulting measurement model, the data y are of the form 

y = FnX + s. (3.1) 

As shown in Figure impulsive corruptions produce additional aliasing that further masks the 
line-spectra locations when we visualize the data in the frequency domain. We aim to demix the 
two components: the sines, whose spectrum is denoted by x, and the spikes s. 


3.1.1 Estimation via convex programming 


Convex programming has proven very effective in separating incoherent low-dimensional compo¬ 


nents 20,25,47,51 . Inspired by these works, we design a simple convex program that allows to 
achieve effective demixing without any previous knowledge of the number of spikes or sines. To 
separate the two components we penalize sparsity-inducing norms in both domains: the £i norm 
of the spikes and the TV norm the spectrum of the sines, which is modeled as an atomic measure. 
These two norms are combined additively. More explicitly, the estimate is computed by solving 


mm X 

X. 


TV 


-|-f?||s||i subject to iFnX + s = y, 


(3.2) 


where r/ > 0 is a real-valued parameter that determines the tradeoff between the two terms in the 


cost function. This formulation was previously proposed in 73 . 


3.1.2 Implementation 


As in the case of our basic model (see Section 2.7), we have two options to solve Problem (3.2). The 
hrst is to discretize the support of the primal variable that represents the line-spectra estimate and 


then solve an ^i-norm minimization problem. The second is to consider the dual of Problem (3.2), 


provided by the following lemma (see Section A.2 of the appendix for the proof) 


Lemma 3.1. The dual of Problem (3.2) is 


max (y, c) subject to 
ceC" 


\Kc\\oo<^ and 


|c|loo < V 


(3.3) 


Problem 3.3 can be recast as the semidefinite program 

A c 


max (y, c) subject to 
ceC", AeC"><" 


1 


^ 0, T* (A) = ei and 


|c||oo < V, 


(3.4) 


by invoking Proposition 2.4 After solving (3.4) we still need to decode the support of the primal 


solution from the dual solution. This can be done by leveraging the following lemma, proved in 


Section |A.3| of the appendix. 

Lemma 3.2. Let 




and 


Si 


= \si\e^^\ l£S<Z{l,...,n}, 
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Figure 6: Demixing of the signal in Figure by semidefinite programming. Top left: the 
polynomial T* c (green) corresponding to the solution of (3.4) c interpolates the sign of the 
line spectra of the sines (orange) on their support. Top right: ri~^c interpolates the sign pattern 
of the spikes on their support. Bottom: locating the support of x and s allows to demix to 
very high precision. 


be a solution to (3.2), such that T 
spikes s respectively. // c G C” is a 


and S are the nonzero supports of the line spectra x and the 
corresponding dual solution, then for any tj in T 


{Kc) {tj) 




and for any I, 1 < I < n in S 


ci=rie^^h 


In words, the weighted dual solution r]~^c and the corresponding polynomial J-^c interpolate the 
sign patterns of the primal-solution components x and s on their respective supports. This suggests 
the following demixing scheme: 


1 . 

2 . 

3. 

4. 


Solve the finite-dimensional semidefinite program (3.4) to obtain a dual solution c. 

The support of s is given by the points at which c equals rj. 

Construct the support-locating polynomial iff c. The support of x is given by the points at 
which its magnitude equals one. 


Estimate the amplitudes of x and s by solving the corresponding system of equations. 
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The resulting x and s are our estimates of the line spectra and of the spikes respectively. Figure 
shows the results obtained by this method on the data described in Figure both components are 
recovered very accurately. 


3.1.3 Theoretical analysis: Dual certificate 


The success or failure of the demixing procedure is determined by the structure of the signal that 
we aim to recover. As in the case of our basic model, this is reflected by the dual certificate 


corresponding to Problem (3.2). 


Lemma 3.3. Let y = FnX + s, where 


X = 


ajSt^ = ^ \aj\ and si = \wi\e 




for s G with support S C {1,... ,n}. If \T\ + |5| < n and there exists a low-pass polynomial 
q:T 


where c G C”, such that 


fc 

q{t)=T*c= Y, 
i=-U 


q{tj)=e^'^^ tj€T, |g(t)| < 1 t^T, 

ci = rie^'^^ I e S, \cj\<rj j ^ S, 


(3.5) 


(3.6) 

(3.7) 


then X and s are the unique solutions to Problem (3.2). 


The lemma, proved in Section A.4 of the appendix, shows that the dual certificate is a low-pass 
polynomial which interpolates the sign pattern of the line spectra, just like the dual certificate corre¬ 


sponding to Problem (2.6). As a result, the approach might fail for line spectra with very clustered 


supports, which makes sense as such problems are challenging even in the absence of corruptions. 
The dual certificate satisfies two additional conditions that depend on the spikes/corruptions: its 
coefficients interpolate the sign pattern of the spikes on their support and their magnitude is 
bounded by y. If there are too many spikes it may become impossible to build a bounded low-pass 
polynomial under these constraints that also interpolates the sign of the line spectra. There is 
consequently a tradeoff between the number of sines and spikes that may be recovered simultane¬ 
ously. In the following section we provide some experimental results that illustrate this tradeoff. 
Characterizing it theoretically is an interesting research direction. 


3.1.4 Numerical results 

Figure shows the result of applying our demixing procedure to signals with different minimum 
separations of the line spectra x, as well as different cardinalities of the supports of x and s. In each 

Matlab script reproducing this example is available at http: //www. cims .nyu. edu/~cfgraiida/scripts/sines_ 
and_spikes.m 
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Number of spikes 


10 spikes |T| = 15 



Minimum separation Amin/Ac 


Minimum separation Amin/Ac 


Figure 7: Graphs showing the fraction of times Problem (3.2) achieves exact recovery over 10 
trials with random signs and supports for different values of the minimum separation of the line 
spectra x (left), as well as different cardinalities of the support of the line spectra x denoted 
by \T\ (center) and of the spikes s (right). The simulations are carried out using CVX 
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case we observe a rapid transition between a regime in which exact recovery almost always occurs 
and a regime in which the method fails. For a fixed minimum separation, the procedure succeeds 
for a number of spikes which decreases as we increase the number of sine components. In contrast, 
if we fix the cardinality of x, the phase transition occurs at a value of the minimum separation that 
does not seem to depend on the number of spikes, as long as there are not too many. Similarly, for 
a fixed number of spikes the minimum separation necessary to achieve exact recovery seems to be 
independent of the cardinality of x below a certain limit. 


3.2 Point sources with a common support 


In this section we consider the problem of super-resolving m signals consisting of superpositions of 
point sources, which share a common support. This is a model of interest in fluorescence microscopy, 
where the data are often well modeled as low-resolution frames where fluorescence probes flicker 
stochastically [9,44 ,^, and in sparse-channel estimation for wireless communications [^. 

The signals are modeled as superpositions of Dirac measures with support T 


Xk{t) 


tjGT 




1 < k < m, 


(3.8) 


where A G The low-resolution measurements consist of the n = 2/c 4- 1 first Fourier 

coefficients of each signal. We represent these low-pass data with cut-off frequency fc as a matrix 
Y G 


Yik= e 




Xk{dt) = Y, Xk)i , \l\<fc.l<k< m. (3.9) 




To be clear, the kth column of Y contains the low-pass Fourier coefficients of the signal 
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3.2.1 Estimation via convex programming 


Note that we could just apply the approach based on TV-norm minimization described in Sec¬ 
tion to super-resolve each signal separately. However, our goal is to exploit the diversity of the 
amplitudes Ajk to obtain a better estimate of the common support T by processing all the data 
simultaneously. This can be achieved by leveraging a generalization of the TV norm, which we call 
group total-variation norm (gTV). Formally, we define the gTV norm as 

||V|| TV— sup f F'k{t)Xkidt) 



We would like to stress that the gTV norm is a well-known quantity. In fact, it is commonly used 
as a definition for the total variation of vector-valued signals [^. Our intent is not to propose 
a new name for it, but rather to use an abbreviation that distinguishes it from the TV norm 
while highlighting the connection to group sparsity. Intuitively, the gTV norm is the continuous 
counterpart of the mixed £i/i 2 norm, which is used to promote group sparsity (TO, 79 . For an 
arbitrary matrix M 


mw/i, ■.= Y.\\^r\\2 

j 


where Mj- denotes its jth row. 


V 


gTV - 


h/f.2- 


If the measures are superpositions of point sources then 


Let us provide some intuition as to why the t\li 2 norm promotes joint sparsity. Consider a matrix 
M which has both zero and nonzero elements. If we choose an element Mjk from an empty row 
Mj: and set its value to e > 0, then increases by e. However, if the row is not empty and 

||Mj :||2 > 0, then increases by just e^/2 ||Mj ;||2 as long as e <C ||Mj :||2 (this follows from 

a Taylor expansion around e = 0 of (||Mj :||2 It is consequently much cheaper to introduce 

new elements in non-empty rows than in empty ones, which promotes a common sparsity pattern 
in the columns. 


To super-resolve the signals jointly, we propose minimizing the continuous version of the £ 1/^2 norm 
subject to data constraints 


min 

V 

subject to 

Xn Xi Xn X2 ■ 

• Xn Xm 

X 


gTV 




(3.10) 


This cost function favors solutions that have a sparse common support, allowing to jointly super¬ 
resolve signals that are very challenging to super-resolve individually, as illustrated in Figure ^ We 
would like to note that this approach is equivalent to the one proposed by two recent preprints 48|78] 
from the perspective of atomic-norm minimization. 


3.2.2 Implementation 


If we discretize the domain. Problem (3.10) reduces to an 7i/72-norm minimization problem. How¬ 
ever, as in the case of Problems (2.6) and (3.2), it is possible to obtain a solution over the continuous 
domain directly. 
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Measurements (signal 1) 


Measurements (signal 2) 


Measurements (signal 3) 




TV-norm minimization (signal 2) 



TV-norm minimization (signal 3) 





Group-TV-norm minimization 


Group-TV-norm minimization estimates 




X Signal 1 
X Signal 2 
X Signal 3 
O Estimate 1 
O Estimate 2 
©Estimate 3 


Figure 8: Top row: Signals and corresponding low-pass data. Middle row: Support-locating 
polynomials obtained via TV-norm minimization. Bottom row: Support-locating polynomial 
obtained by gTV-norm minimization (left) and the resulting amplitude estimates (right, the 
three signals and their estimates are plotted simultaneously). 


Lemma 3.4. The dual of Problem (3.10) is 


max {Y,C) subject to sup |(J^* Cfc) (t)|^ < 1, 

CeC"x™ t 

k=l 

where Ck denotes eolumn k of C. 

The proof of the lemma can be found in Section A.5 of the appendix. By Proposition 2.4 
recast (3.11) as the semidefinite program 

{Y, C) subject to 


max 


A C 
C* In 


Y 0, T* (A) = ei, 


(3.11) 


we can 


(3.12) 


19 






































































































































































































where 4i is the identity matrix of dimensions n x n. 

To recover the support of the primal solution from a dual solution C we compute a generalization 


of the polynomial (2.17) defined in Section 2.7 


(3.13) 


fc=l 

The following lemma establishes that we can use this polynomial to locate the support of the primal 


solution. We defer the proof to Section A.6 in the appendix. 


Lemma 3.5. Let X be a solution to Problem (3.10) such that Xk{t) = Xlt cf 1 < k < m, 


where A G 


T xm 


and T denotes the support of the solution. Then, for any corresponding dual 


solution \C\, the support-locating polynomial satisfies 

Ppitj) = 1 for all tj G T. 

The careful reader will observe that the lemma does not imply that the support-locating polynomial 
cannot be equal to one all over the support and hence completely uninformative. However, in the 
proof of the lemma we show that the polynomial X* C^, which is restricted to the hypercylinder 
of radius one by constraint (3.11), is equal to Ajk/ ||Aj :||2 on T (recall that Aj-, is the jth row of 


A). This means that except in pathological cases where the polynomial stays on the surface of the 
hypercylinder over all of its domain, the support-locating polynomial is informative. Indeed, this 
is what we observe in numerical simulations. See Section 4 of for a related discussion. 

Finally, the procedure to estimate the signals is as follows. 


1. Solve the finite-dimensional semidefinite program (3.12) 


2. Construct the support-locating polynomial (3.13). Set T to the set of points where its mag¬ 
nitude is equal to one. 

3. Estimate the amplitudes of the individual signals by solving the corresponding systems of 
equations. 

Figure [8 shows the polynomial Pq for an example where m = 3, s = 15, /c = 40 and the signal 
amplitudes are real valued and generated at random. The minimum separation of the common 


support is 0.7Ac. The polynomial P^ obtained from (|3.12|) allows to locate the support of the 


signal with great precision. In contrast, the individual support-locating polynomials obtained from 
solving the individual TV-norm minimization problems do not allow to locate the support: they 
spikes and also produce false positives. 


miss some 


3.2.3 Theoretical analysis: Dual certificate 


As in the case of Problems (2.6) and (3.2), the dual certificate corresponding to Problem (3.10) 


provides some useful intuition as to the performance we can expect from the optimization-based 
method depending on the structure of the signals that generate the data. 


Matlab script reproducing this example is available at http://www.cims.nyu.edu/~cfgrcinda/scripts/ 
superres_common_support.m 


20 
















Lemma 3.6. For X defined by (3.8) and Y defined by (3.9), if there exists an m-dimensional 
low-pass polynomial Q : T —)■ C”^, 


/c 


Quit) = iKCk) it) = Y. 

i=-F 


i2nlt 


1 < k < m, C = 


Cl ••• c„ 


such that 


Qkitj) 


A 


jk 


tj £ T, 


max 

t£T' 




k=l 


then X is the unique solution to Problem (3.10). 


The dual certificate provided by this lemma (see Section A.7 in the appendix for the proof) is 
an m-dimensional low-pass polynomial Q that interpolates points from the m-dimensional unit 
hypersphere 5"* on the support of the signal while remaining strictly inside the hypercylinder 
T X 5"* elsewhere. If the amplitudes of the m signals are in a general position, then it is plausible 
that interpolating these points may become easier as the dimension m of the hypercylinder increases. 
This would imply that using a larger number of signals may allow to recover common supports with 
smaller minimum separations. This is confirmed by the numerical simulations in Figure which 
show that exact recovery occurs for decreasing values of the minimum distance as we increase m. 


3.2.4 Numerical results 

Figure shows the fraction of successes achieved by gTV minimization for different values of the 
minimum distance, of the number of signals m and of the number of point sources in each signal |r|. 
As in the case of TV minimization for a single signal (see Figure]^, a phase transition occurs at a 
certain value of the minimum distance. This critical value of the minimum distance decreases as m 
increases, saturating at Ac/2. Below this value, there exist signals that may lie almost in the null 
space of the measurement operator (note that this is a stronger statement than the difference of two 
signals being almost in the null space, see Section 2.2). Characterizing the minimum separation 


at which gTV-norm minimization succeeds for amplitudes in a general position is an intriguing 
research problem. 


4 


Proof of Proposition 


2.3 


Our aim is to construct a low pass polynomial that interpolates an arbitrary sign pattern v £ 
on a set of points T separated by a minimum distance of 1.26//c. As a first idea, we could consider 
interpolating the sign pattern with a Dirichlet kernel, 


Kif,t) 


1 

2 / + 1 


/ 

E 


^i2'Kkt 


k=-f 
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m = 1 (Real amplitudes) m = 1 (Complex amplitudes) 




0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 

Minimum separation Aniii,/A(. Minimum separation Amin/Ac 



m = 2 (Complex amplitudes) m = 10 (Complex amplitudes) 



Minimum separation A^iJA, Minimum separation A„iin/Ac 


Figure 9: Graphs showing the fraction of times Problem (3.10) achieves exact recovery over 
10 trials with random signs and supports for different values of m. As m grows the minimum 
separation at which exact recovery occurs tends to Ac/2. The simulations are carried out using 
CVX (4^. 


which can also be written as 


K{f-,t) — < sin((2/+l)7rt) 


(2/H-l) sin(7rt) 


if t = 0 
otherwise . 


(4.1) 


The result of the interpolation would satisfy (2.8) and (2.9) by construction, but controlling its 


magnitude on the off-support would be problematic because of the slow decay of the tail of K. An 
alternative is to use a faster-decaying kernel instead, as in |18| which uses the fourth power of the 
Dirichlet kernel, also known as Jackson kernel. Taking the fourth power of a kernel is equivalent to 
repeatedly convolving it with itself in the frequency domain, which smooths out the discontinuities 
of its spectrum. The resulting kernel is consequently not as spiky at the origin, but has a faster 
decaying tail. Generalizing this idea, we can take the product of p kernels with different bandwidths 
life, 72/0 • • ■) Ipfc to achieve a better tradeoff between the behavior of the resulting kernel at the 
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Spectrum 


K-y{t) 




Figure 10: Interpolating kernel used in the proof (left), along with its asymptotic decay 
(center) and its low-pass spectrum (right). It is the product of three Dirichlet kernels with 
cut-off frequencies 0.247 fc, 0.339/c and 0.414/c. 


origin and the decay of its tail. This yields a kernel of the form 

P fc 

(t) :=l[K (7,/c, t) = E 

i=i fe=-/c 


(4.2) 


where c G C"' is the convolution of the Fourier coefficients of K ( 71 /c, t ), K ( 72 /c, t),... K (7p/c, t). 
As long as Ij — cut-off frequency of the kernel is indeed fc- We fix p = 3 and 71 = 0.247, 


72 = 0.339 and 73 = 0.414. Figure 10 shows and its spectrum. 


Interpolating the sign pattern with an adequately selected kernel is not sufficient to produce a 
valid construction. The reason is that the magnitude of the polynomial will tend to exceed one 
near the elements of the support T. This can be avoided, however, by forcing the derivative of 
the kernel to be zero at those points, an idea introduced in 18 . As a result, the magnitude of 


the polynomial has a local extremum at the interpolation point and obeys condition ( 2 . 11 ) for a 


sufficiently large minimum separation, as illustrated by Figure 11 In order to enforce the extra 


constraint on the derivative of q, we need more degrees of freedom in the construction. For this 
purpose, we incorporate the derivative of the kernel, so that the construction is of the form 


q{t) = E ajK.y{t - tj) + - tj), (4.3) 

tj£T 

where a, (5 ^ are coefficient sequences satisfying 

9 ( 4 ) = E ~ (4 - 4) = tk G T, (4.4) 

tj&T 

9j?^(4) + iqf\tk) = E (4 - 4) + (4 - 4) = 0, 4 e T, (4.5) 

tj&T 


and qR and qj are the real and imaginary parts of q. 
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Figure 11: Dual-polynomial if we only use to interpolate the sign pattern (left), as 
opposed to also incorporating its derivative into the construction and forcing the derivative of 
the polynomial to be zero at the interpolation points (right). 


This construction trivially satisfies (2.8) because both and are low pass. The follow¬ 
ing lemma, proved in Section 4.2, establishes that (2.9) holds and derives some bounds on the 
interpolation coefficients. 


Lemma 4.1. Under the hypotheses of Theorem 2.2. there exist coefficient vectors a and (3 obey¬ 
ing (4.4) and (4.5), such that 


||a||^ < := 1 -h 2.370 10“^ 

II/3IL </3^:= 4.247 10-2 Ae. 

In addition, if vi = 1, 

Real > := 1 - 2.370 10-2, |Imai| < af := 2.370 10-2. 


(4.6) 


(4.7) 


The proof is completed by two lemmas, proved in Sections 4.3 and 4.4, which show that our 


construction satisfies (2.10) and (2.11) 


Lemma 4.2. Fixto G T. Under the hypotheses of Theorem 2.2. |g(t)| < 1 for\t — to\ G (0, 0.288316 Ac 
and there exist numerical constants Cq G (0,1), Ci and C 2 such that (2.11|) holds. 


Lemma 4.3. Fix to G T. Let be the nearest element of T such that > to ^nd t- be the 
nearest element of T such that t_ < to. Under the hypotheses of Theorem 2.2. |g(t)| < 1 for 
to — <t<to o.iT'd to < t < to -\- *'*'2 • 


The proofs of Lemmas 
Section 14.11 

The proofs of the different lemmas required to establish our main result involve some rather intricate 
calculations. In order to ease the task of checking the proof, we provide a Matlab script that 
performs these calculations at http: //www. cims.nyu.edu/~cfgranda/scripts/superres_proof . 
zip, 


4.1, 4.2 and 4.3 make heavy use of bounds on K-. that are presented in 
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4.1 Bounds on the interpolation kernel 


Our results rely on sharp non-asymptotic bounds on the Dirichlet kernel K (4.1) and its derivatives, 
which can in turn be used to establish bounds on the interpolation kernel (4.2) and its deriva¬ 
tives. To avoid excessive clutter we defer the precise expressions and their proofs to the appendix. 
The following lemma, proved in Section [C.l provides upper and lower bounds near the origin. 

Lemma 4.4. For fc > 10^ and i G {0,1,2,3}, for any t in the unit interval if r is such that 
\fct — t\ < e then 

(r) - ( 27 r)'+i (t) < (t) + ( 27 r)^+i /fe 

where Bf p and Bffp are defined in Section 


B.l 


of the appendix. 


A direct corollary yields a sharp bound on the magnitude of the kernel and its derivatives. 

Corollary 4.5. For fc > 10^, i G {0,1,2,3} and any t in the unit interval, if t is such that 
\fct — t\ < e then 

{t) < (r, e ) := max (r)| , ('r)|} + (27r)^+^ f^e. 

Away from the origin, we establish bounds on the absolute value of K.y and its derivatives that are 
weaker, but decreasing. The following lemma is proved in Section C.2 of the appendix. 

Lemma 4.6. For fc > 10^, £ G {0,1, 2, 3} and |t| < ^ 

where b.y^i are decreasing functions defined in Section B.S\ of the appendix. 

Corollary |4.5| and Lemma 4.6 are combined in the next lemma, proved in Section [C.3t to bound the 
sum of absolute values of shifted copies of the kernel (and of its derivatives) if they are separated 
by the minimum distance. The lemma is stated for an interval around the origin, but it holds for 
any position in the unit interval. 

Lemma 4.7. Assume 0 G T and let Tmin := Amin/Ac = 1-26 denote the minimum separation of 
the set T, sealed so that A (T) > Tcamj fc- If fc > 10^ and 7 = [0.247,0.339,0.414]^, for any 
0 < t < Amin/ 2 , e > 0 and r such that t — e < fct < t 

^ K\^Ht-tfi <77,(r) + i7,(-r). 


bS'r\{0} 


where 


M ■= y] max ^ max B^^ (u, e), {{j + 4) Tmin) ^ + Q, 

where Qj^r is an e-grid of equispaced points covering the interval [jTmin — t, (j + 4) Tmin] o,nd 


400 


Cl {{j ~ 1/2) Tmin) + Cf, 

j=21 

Co = 8.852 10-^ Cl = 5.562 10"®, C 2 = 3.495 10"^ C 3 = 2.196 10"^. 
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4.2 Proof of Lemma 14.11 


Set 


{Do)^, = K^{tj-tk), (Z)i),, = i^W(t, -4), (Z)2U = i^f (ii-4), 


Jjk 


jk 


where j and k range from 1 to |T|. With this, (4.4) and (|4.5[) become 


'Do 

Di 


a 


V 

Di 

D2 


P. 


0 


By standard linear algebra result, this system is invertible if and only if D 2 and its Schur complement 
Do — DiD^^Di are both invertible. To prove that this is the case we can use the fact that a 
symmetric matrix M is invertible if 

\\I-M\\^<1, (4.8) 

where ||^||oo is the usual infinity norm of a matrix defined as ||^||oo = niax|| 3 ,||^^]^ H^dxlloo = 
maxj \aij\. This follows from M~^ = (/ — H)~^ = X]fc>o H = I — M, where the series is 
convergent since \ \H Hoc < 1- In particular. 


II" 


-i| 


< 


1 




By Lemma 4.7 


/—DqI loo — {Ky {to — tj)\ < 2Ho (0) < 1.85510 

tosT ^ ^ 


tj&T\{to} 


||T)i||^<max ^ K^^^to-tj) < 2 iLi ( 0 ) < 0 . 1110 /c, 

toST 


tj&TXito} 


k\^Ho)I-D 2 <max V (to - tj) < 2 H 2 (0) < 1.895 fI 

00 to^T ' 

tj GT\{to} 


(4.9) 

(4.10) 

(4.11) 

(4.12) 


( 2 ) 

Note that D 2 is symmetric because the second derivative of the interpolation kernel Kj ' is sym- 

D 2 


metric. The bound (4.12) implies 


I- 


( 0 ) 


< 1, 


which implies the invertibility of D 2 . To control the infinity norm of the inverse of D 2 we need 

( 2 ) 

bounds on the value of K!^ ' at the origin. 

Lemma 4.8. If fc > /min 


4vr272/2 


JJC 


77^^^ ( 0 ) < < 0 , where ^ 5°’^ ( 7 ^) and 5°’^ ( 7 ^) 

i=i 

Proof. The result is a consequence of K (/, 0) = 1, (/, 0) = 0, (/, 0) = — 47 r^/ (1 + /) /3 

and (C.19). □ 
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By the lemma and (4.9), we have 


D. 


-i| 


< 


< 


1 


( 0 ) 


( 0 )/-L >2 

1 


B. 


o,u 

7,2 


{0)I-D2 


< 0.3738 A^. 


Combining this with ( |4.10[ ) and ( 4.11| ) yields 

||/_ (^Do - D,D^^D,)\\^ < ||/- Z^olloo + ll^ill 

< 2.315 10“^ < 1. 


2 

oo 


\D. 


-i| 


(4.13) 


(4.14) 


Note that the Schur complement of D 2 is symmetric because Dq and D 2 are both symmetric 
whereas DJ = —Di since the derivative of the interpolation kernel is odd. This shows that the 
Schur complement of D 2 is invertible and, therefore, the coefficient vectors a and /3 are well defined. 

There just remains to bound the interpolation coefficients, which can be expressed as 


a 


I 

P. 


-DpD, 


C-\, C := Do- DiD^^Di, 


where C is the Schur complement. The relationships (4.9) and (4.14) immediately give a bound on 
the magnitude of the entries of a 


\a\ 


= l|c- 7 |u<||c 


-i| 


< 1 + 2.370 10 


-2 


Similarly, (4.11), (4.13) and (4.14) allow to bound the entries of /3: 


II/5|Ioo<II^2-'^iC^ 


',-11 


-ll 


— I \ 


ll^llloollC^ 


-ll 


< 4.247 10“^ A^. 


Finally, with vi = 1, we can use (4.14) to show that ai is almost equal to 1. Indeed, 


|1 — ail = |l — (C 

= |(7-C-7),| 

< lU-c-'ll 

— M M 00 

= l|C-‘(7-C)||^ 

<I|C-‘|L l|7-C|L< 2.370 10-7 

This concludes the proof. 


4.3 Proof of Lemma 14.21 

Without loss of generality we map the unit circle T to the interval [— 5 , 5 ] and we set to = 0 and 
vq = 1. In order to bound the magnitude of the dual polynomial locally, we control the second 
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derivative of its square around the origin. In particular, we bound it on an interval of the form 
fct G [je, (j + 1) e] for an integer j > 0 and e > 0. Let us decompose q = qR + iqi into its real and 
imaginary parts. The derivative is equal to 

(t) = 2 (^R (t) q^^'> (t) + qi {t) qf^ {t) + (O) + ( 4 ^^ (^)) 

= 2 (^qR (t) (t) + qi (t) ^ (t) + \qi , 

where qi {t) := + iq\^\ To alleviate notation we define 

Gej,e (oi, 02 ) := (je, e) + 02 {Hi (— {j + 1 ) e) + Hi {{j + 1) e)), 

where is defined in Corollary 


4.5 


By Lemmas 


4.1 


4.4 


and 


4.7 


and Corollary 


4.5 


(4.15) 
we have 


(^) = X] -tj) + Re il3j) {t - tj) 


> Re (ao) (t) - ||a||(^ ^ \K^{t-tj 


^ {t - tj 


> (R^o (j 6 ) - 27re) - (_ + 1) e) + Ho {{j + 1) e)) - Gg,-,, (/l^, /^U) 

■= Jo,R 0 ) ) 

for je < fct < {j + 1) e since is positive in the interval of interest. Similarly, 


\qR{t)\ < Go,j,e {a^,a^)+Gi,j,, {P^, 

\qi (i)| < Go,,,. (aF,aU) +Gg,-, 


= Jo,R (j) , 
= Jo,I (j) , 
= Jl (j) , 


\qi (i)| < Gi,,,. (aU, aU) + ^2,,,. {p^, P^) 

q^R^ (t)| < G2,,,. (aU,aU) + ^ ^ 

qf^ (t)| < G2,,,. (ay,aU) + G3,,,. {P^,P^) := Js,/ (j), 

and if B }^2 {h) + (27r)^ e < 0 

q^R (t) < aF (je) + ( 27 r)^ (772 (— (j + 1) e) + H2 {{j + 1) e)) + Gs,,-,. (/ 3 ’^, / 3 ^) := (j) 

Combining these expressions we obtain a bound on the second derivative of the square of the 
polynomial q, 





2 f ^ 2 T O') 0) + '^24 0) Jo, I 0) + Jl of 

2 ( J2,iJ (j) J(Fi? (j) + J2,I (j) Jo,/ (j) + Jl (jf 


if O'e) < 0 

otherwise. 


(4.16) 


which holds for any t such that fct G [je, {j + 1) e]. This bound allows us to control the curvature 
of the squared magnitude of |g| on a fine grid, which can in turn be leveraged to obtain a bound 
on 1^1. This is made precise by the following lemma, proved in Section C.5 of the appendix. 
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Lemma 4.9. If F is a twice continuously-differentiable real-valued function such that 


F ( 0 ) = 1 , ( 0 ) = 0 , sup F( 2 ) (f) < F 2 j, 

ie<t<(i+l)e 

for a certain positive real number e and all nonnegative integers j > 0 , then if ke < t < (k -\-1) e 
for an integer k we have 

F (t) < F^ := 1 + eM E + lF,>oi "2 j , 


where 


k-l 


F2 = '^F2,j + lF2,k>oF‘ 

3=0 


2,k- 


We apply the lemma by setting F = \qf. By Lemma 4.1, F (0) = |q'(0)|^ = 1 and (0) = 
(0) (0) + (0) (0) = 0. To obtain a bound on |g|^ we evaluate the bound computa¬ 

tionally on a grid with step size e = 10“® covering the interval [0,0.3]. The corresponding values 
of Ff° are plotted on the left of Figure 


12 


Since F“ lies below 1 — 6.394 10 ^ between k = 1 and 


k = 0.288316/e, we have 

k(t)| < 1 - 3.19710"'^, < 0.288316 Ac, 4 e T, 

since we fixed to = 0 without loss of generality. 

For t belonging to the interval (0, e) we apply the following inequality, which follows from the 
fundamental theorem of calculus and the facts that F' (0) = 0 and F 2 fi < — 3.23/c (the proof of 
Lemma 4.9 is based on a similar argument) 


sup \q{t)\ < sup F (0) +sup F" (u) 

iS(0,e) te(0,e) nS[0,e] 

= sup 1 + F2 ,o t^ 
ie(0,e) 

< sup l-2,.22,flt^ 

46 ( 0 , 6 ) 

< 1 . 


Finally, to establish the quadratic bound (2.11) on the magnitude of q we prove that in the interval 


the second derivative of |( 7 | lies strictly below zero. In fact, we can make use of the bound (4.16) 
Since 

d2 \q\ {qR it) it) + qi it) (t))' ^ (i) (i) + (t) gf) (t) \g^ (i)|2 


dF |g(f)|3 

[qr it) Qr^ it) + qi it) q\^^ (t)) 


\qit)\ 


2 |g(t)| dF 
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Bound on \q\^ 


Bound on 




Figure 12: On the left, plot of obtained by applying Lemma 4.9 with F = \q'f'. The 
bound is evaluated on a grid with step size e = 10“® covering the interval [0,0.3]. Between 
10“® and 0.288316 it remains strictly below 1. On the right, plot of F 2 j ( |4.16| ) evaluated on 
a grid with step size e = 10“® covering the interval [0,0.2]. Between 10“°and 0.110497 the 
bound is strictly negative. 


this second derivative is negative if is negative. To bound this latter quantity we evaluate 

the bound F 2 J (4.16) on a grid with step size e = 10“® covering the interval [0,0.2]. The result is 


plotted on the right of Figure [T2| Between 0 and 0.110497 the bound is strictly negative, which 
immediately implies that this is also the case for the second derivative of jqj. This establishes the 


upper bound in (2.11), because q (to) = 1 and the derivative of \q\ is zero by Lemma 4.1 To prove 


the lower bound we just need to bound the second derivative of the real and imaginary parts of 
q — Vj. Since the argument is almost identical to the one used to bound the second derivative of 
\q\ we omit the details. 


4.4 Proof of Lemma 14.31 


By Lemmas 4.1, 4.4 and 4.7 and Corollary |4.5| 

k(t)| < Go,,-., 


(4.17) 


for any t < Amin/2 in [je,{j + l)e], where is defined in ( |4.15| ). We evaluate the bound 

numerically on a grid with step size e = 10“® to establish that in the interval [0.288316, rmin/2] 

\q{t)\ < 1-4.64110-3. 

The numerical values are plotted in Figure [T^ 


Together with Lemma 4.2, this completes the proof except for the following case: The first spike 
to the right of to, which we denote by ti, might be at a larger distance than Amin, so it is still 
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Figure 13: The bound (4.17) is evaluated on a grid with step size e = 10 ° covering the 
interval [0,0.3]. Between 0.288316 and ri„in/2 it remains strictly below 1. 


necessary to bound q in the interval [Amin/2, to + • Let t be a point in this interval, then 


kWI = 


^ ajK^ {t -tj) + {t - tj) 


< ^max |maxB“o {u, e), b^^o (2rmin)| + Hq (rmin/2) + Hq (0)^ 

+ ^max |max5“i {u, e), (2Tmin)| + Hi (rmin/2) + Hi (0)^ , 
where ^ is a grid with step size e covering the interval [rmin/2, 2Tmin]- The bound follows from 


Lemmas 4.1 4.4 and 4.7 and Corollary 4.5 In particular, from the definition of Hi in Lemma 4.7 
it is not difficult to show that if rmin /2 < t < ti — then 

^ I iLW (t - ti) I < Ht (rmin/2) and | AT^ (t - ti) | < iL^O) • 

beTn(o,|) 4.6Tn[-|,o) 

Finally, we evaluate the bound numerically for a step size of e = 10“® to prove that 

|g(t)| < 0.896 

in [Amin/2, to+ L^]. 
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5 Conclusion and future research 


In this work we provide a reasonably tight characterization of the performance of super-resolution 
via convex programming when the signal consists of point sources in one dimension that are sepa¬ 
rated by a minimum distance. Interesting research directions include developing a sharper bound 
than the one provided by 18 for two-dimensional signals, studying other conditions that allow for 
a small degree of clustering in the support of the signal as in and extending the analysis to 
other signal models such as piecewise-constant signals in two dimensions. We have also derived 
optimization-based algorithms to tackle the demixing of sines and spikes and the super-resolution 
of point sources with a common support. In both cases, it would be interesting to derive theoretical 
guarantees by building the appropriate dual certificates, as outlined in Sections [3.1.3 and 3.2.3 Fi¬ 
nally, applying optimization-based approaches in real applications will require developing efficient 
algorithms to solve the corresponding optimization programs on large-scale datasets (see |15| for 
recent work in this direction that is relevant to super-resolution). 
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A Proof of auxiliary results in Sections and 

A.l Proof of Proposition [274] 

Let us define 


zo{t) := 
zi{t) := 


J2TTfct i27r(/c-l)t 


„-i2TTfct 


1 T 


1 


i2'Kt 


A27Tnt 


iT 


Note that {T*Ck){t) = zo{t)*Ck, so that 
A — CC* ^ 0 and T* (A) = ei. This implies 


zo{f)*CC*zo{t). If (2.15) holds then 


Y,\{KCk){tt = zo{trCC*zo{t) 

k 

< zo{t)*Azo{t) 

= {zo{t)*Azo{t)) 

= zo{t)zo{t)*) 

= {^,T{zi (t))) 

= {T* (A), it)) 

= = 1 . 


For the converse, [T"* Cfc|^ < 1 implies that the trigonometric polynomial 1 — zo(t)*CC*zo{t) is 
non-negative. By the Fejer-Riesz Theorem there exists a polynomial P{t) = c*zo{t) such that 

1 - zo{t)*CC*zo{t) = \P{t)\‘^ = zo{t)*cc*zo{t). (A.l) 
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Now let us set A = CC* + cc*. A and A — CC* are positive semidefinite by construction, which 
implies 


Furthermore, by (A.l) 


A 

C* 


C 

I 


>1 0 . 


1 = zo{t)*Azo{t) 

= {Kzo{t)zoit)*) 
= {A,T{z,m 
= {T*{A),zi it)) 


for all t, which is only possible if T* (A) = ei. This completes the proof. 


A.2 Proof of Lemma 13.11 

The Lagrangian is equal to 

L {x, s,c) = \ \x\\rj,y + rj \ \s\\^ + {c,y - J^n X - s) 

= {c,y) + llxllrpy - {J='*c,x) + ri\\s\\-^ - {c,s) 

where c G C” is the dual variable. The lower bound on the cost function provided by minimizing 
the Lagrangian over x and s is trivial (—oo) unless |(T'*c) (t)| < 1 for all t and ||c||^ < rj. Under 
these constraints, the minimum is attained for x = 0 and s = 0 for any value of c. Maximizing the 
remaining expression over c yields the dual problem. 


A.3 Proof of Lemma 13.21 


The interior of the feasible set of Problem (3.4) contains the origin and is consequently non empty, 
so strong duality holds by a generalized Slater condition and we have 


= II^IItv + ^IIs|Ii = (c,y) 

tjST leS 

= (c, JZnX + s) 


= Re 




les 


By Holder’s inequality and the constraints on c, this implies the result (otherwise equality would 
not be possible). 
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A.4 Proof of Lemma 13.31 


For any vector v and any atomic measure v, we denote by and the restriction of v and i' to 
the subset of their support indexed by a set P {vq has the same dimension as v and t'Q is still a 
measure in the unit interval). Let us consider an arbitrary solution x' and s', such that x' ^ x ox 
s' ^ s. Due to the constraints of the optimization problem, x' and s' satisfy 


y = J^nX + S = Tnx' + s'. 


(A.2) 


The following lemma proved in Section A.4.1 below establishes that and s'gc cannot both be 
equal to zero. 

Lemma A.l. If {x', s'} is feasible and x'rpc and s'^c both equal zero, then x = x' and s = s'. 


This implies that x'j^c and s'^c cannot be a solution, since 

I Itv + ^ I kill “ I I^tI Itv I Itv + ^ I ksl li + ^ I li 
^ I Itv ^ I ksl li + '^ 5 '=) 

> (^Q, x') + (^c, s') by (3.6) and (3.7) 

= {Kc,x') + {c,s') 

= (c, Jnx' + s') 

= (c, J^nX + s) by ( |A.2[ ) 

= {Kc, x) + (c, s) 

= {Q^x) + {c,s) 

> II^IItv + ^ll®lli 

A.4.1 Proof of Lemma lA.ll 

If x'rpc and s'gc both equal zero, then 


(A.3) 


by Lemma A.4.1, (3.6) and (3.7) 


by (3.6) and (3.7) 


(A.4) 

(A.5) 

(A.6) 

(A.7) 

(A.8) 

(A.9) 

(A.IO) 


T"X ~t“ <5 — J~yiXrp — s^ — J~yi X s — J~ n x^p — sQ by (A.2 ) 

= IF n Xrpc + s'gc 
= 0 . 


(A.ll) 

(A.12) 

(A.13) 


We index the entries of S := {zi, ^ 2 , • • ■, *|s|} and define the matrix 

A27rlt 


Ft Is 


If |T| + IjSI < n then 


(Ft);,- = for 1 < / < n, 1 < j < |r| , 

ij 

is full rank (this follows from the fact that Ft is a submatrix of a 


{Is)ii = { n for 1 < / < n, 1 < j < |S| 

U otherwise 


g ([^nx(|r|+|5|)^ where 
(A.I4) 
(A.15) 


Ft Is 


Vandermonde matrix). Equation (A.13) implies 


r 1 

a — a' 

Ft Is 

Vss - Vss'_ 


= 0 , 


(A.16) 


38 


























where Vsu' G is the subvector of u' containing the entries indexed by S and a' G is the vector 
containing the amplitudes of x' (recall that by assumption = 0). We conclude that x = x' and 


s = s 


A.5 Proof of Lemma 13.41 


The Lagrangian is equal to 


£ = 


A 


gTV 


+ (c,T-[a„Ai An As ••• A 


n 


= {C,Y) + 
= {C,Y) + 


X 


X 


gTV 


gTV 


J2(Ck,J^nX, 


k=l 

m 


k=l 


Y,{KCk,Xk 


m ^ ^ 'I ^ r /* 

= {C,Y)+ sup VRe / i^Afc(dt) -VRe / A* (t) A^ (dt) 

||F(t)||2<i^ Ut J ^ Ut 

where C G is the dual variable. The lower bound on the cost function provided by minimizing 

the Lagrangian over A is trivial (—oo) unless 

m 

Y,\iKCk){t)\^<i 


k=l 


for all t. Under these constraints, the minimum is attained for A = 0. Maximizing the remaining 
expression over C yields the dual problem. 


A.6 Proof of Lemma 13.51 

Let us define the matrix P G 

p,k := [KCk) ih). 

The result is an immediate consequence of the following equality 

A. 

An = 


where Mj- denotes the jth row of a matrix M. 

By strong duality, which holds by a generalized Slater condition |57] we have that 


(A.17) 


A 


gTV 


On the one hand. 


A 


gTV 


= {Y,C). 


\T\ 
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On the other, 


{Y,c) = j2{yk,Ck) 

k=l 

m 

= Y,{J^nXk,Ck) 

k=l 

m 

= Y,{Xk,KCk) 

k=l 


1^1 

y~i {ij) I 

i=i fc=i 
|T| 

T,(^r:Pr)' 

i=i 


This combined with the following lemma implies (A. 17). 

Lemma A.2. Let M, N G ^nixn 2 ^ such that the rows of N have non-zero norm, if 


ni n\ 

1=1 1=1 

max \\Mj,\\ <1 

i<3<m 


(A.18) 

(A.19) 


then 


Mjk = 


N, 


jk 


WNy.lW 


Proof. By (A.19) and the Cauchy-Schwarz inequality, {Mj.,Nj-) < ||Aj ;||2 for all j. If for any of 
them {Mj.,Nj-) < 11^^:112 then ( A.18[ ) cannot hold, so 


{Mj,,Ny) = \\Ny\\.^ for all j. 
This directly implies the desired result because for any j 


(A.20) 


M.,- 


l|iVy:| 


~V'’- Hit'll 

= ||A/,J| 2-1 = 0 . 


□ 
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A.7 Proof of Lemma 13.61 


Any m-dimensional measure that is feasible for Problem (3.10) can be written as X + H, where H 


is an m-dimensional measure such that J-nH^ = 0 for 1 < A; < m. This implies that 


{Q, H) = Y, {Qk, Hk) = Y, (KCk, Hk) = Y {Ck,XnHk) = 0. 


(A.21) 


fc=i 


k=l 


k=l 


For any m-dimensional measure X composed by m measures X^ with bounded TV norm supported 
on the unit interval T, we denote its restriction to a subset 5 of T by Xs and its restriction to the 
complement of S by V^c. Since ||Q(t )||2 < 1 for any t, we have {Q,X + Ht) < ||V -|-l^rpy. 
Similarly, since ||Q (t )||2 < 1 for any t in < llTrycUgrpy. This combined with the fact 

that the gTV norm is separable and that by construction {Q,X) = ||V||grpy allows us to conclude 
that 


II A -I- TAjlgrpy _ ||A1 -|- TAyllgrpy -|- | | TI'T’C | | grpy 
> {Q,X + Ht) + {Q,Htc) 

= {Q,X) + {Q,H) 

= ll-^IIgTV > 

as long as is non zero. Since the measurements are injective on T (the restriction of the 
measurement operator is a submatrix of a Vandermonde matrix and thus full rank) = 0 
implies H = 0 and the proof is complete. 


B Bounds on the interpolation kernel 


B.l Upper and lower bounds near the origin in Lemma 4.4 


We use /min to denote a lower bound on the cut-off frequency fc- The letter U denotes an upper 
bound and the letter L denotes a lower bound. The bounds on are 


(r) := ^ m^in^^ ’ 


(B.l) 


1=1 


where {L,U}^ denotes the set of strings of length p where each character is equal to U or L. 
and for £ = 0,1, 2,3 are bounds on the Dirichlet kernel K and its derivatives which are defined 
below. The bounds on K^\ k!^'^ and K!f\ which follow, also depend on these functions. 

^7,1 (^) •= n , 


2 = 1 


^7,1 wn 

i=l ’ j=l,j7^i 


(B.2) 
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^^2 (^) := ^ max (7^, r) (7^-, r) 

. AE|L,U| . . 

1=1 

i=l j=l,j^i ^ ’ k=l,k^{i,j} 


^7,2 W -Z min ^(7i,r) [] 

j=l ’ j=l,j7^i 


+ Z Z UF n ’ 

i=l j=l,j^i ’ /i;=l,/c^{2,_7} 


(B.3) 


i=l ’ j=l,j¥^i 

+ 3 ^ Z ^ max (7* , r ) Bf^ {jj , r ) ]J (7^ , r) 

+ Z Z Z ^max (7i,r)Bf^ (7j,r)Bf'= (7fc,r) [| B^^{-fi,T), 

i=l j=lj^i k=l,k^{ij} ’ /=l,/^{2,_7,/i;} 

^ 7 , 3 wn 

i=l ’ j=l,j¥^i 

+ 3 min B^^ (7*, r) ^ (7^-, r) ]J B^’^ {^k,T) 

+ Z Z Z ^ min ^ (7i,r)Bf^'(7j,r)Bf'= (7fc,r) [J B^‘{-fi,T). 

i=l j=lj^i k=l^k^{i,j} ’ /=l,/^{2 j,/c} 

(B.4) 


These bounds are plotted for 7 = [0.247,0.339, 0.414]^ 


in Figure 


14 


We separate the bounds on K, Bq and Bq, into two intervals. 


^Jiar,o (70, r) if 0 < |r| < To (70) 
^far,0(70,1-) if |r| > ro( 7 o), 


(B.5) 


and do exactly the same for Bq. The intervals depend on the value of 79. In particular: tq (0.247) = 
0.9112, To (0.339) = 0.6615, tq (0.414) = 0.5401. 
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The bounds near the origin are of the form 


-®near,0 (TO; '^) • 


1 - 


27r^7gT^ 27^17^4 / ^ I 1 


15 


1 + 


270/ir 


315 2835 ' 


270/min J sin (27r7or) 


J- o 


^neaifl (TO; '^) • 

Farther from the origin, 

^far,o (70, r) := 


n_2 2 2 / 1^2 

1 - Y (1 + 

3 V 270/m 


'Tf 0 

J 

47r®7QT' 

15 

315 

1 27rSnV" 

~t~ 1 tr 

4vr®7g- 

01 tr 


_ TT^r^ 


6/, 


2 

min 


1 + 


270/min y sin(27r7or) 


^ 277^77^ 277^77'^ 47r®77® 277*77® 

J- 2 r Jg 21 7; T 


315 


2835 


27r7or 


27r7oT ’ 

(B.6) 

(B.7) 


I 1 _ H TT T' 

sin(27r7or) 7 ~ 


27r7or 


*-sin(2777oT)<0 27o/, 


^far,o (70, r) := 


sin (27r7or) lsin(2777or)>0 2/^ 

27r7or 7^1 


min 

.2^2 


+ 1 


cos (27r7or) 

cos(2.7or)>0^^27o/7„in 


+ 1 


cos(27r7oT)<0 


cos (27r7oT) 

1 T 270/min 


(B.8) 

(B.9) 


‘■sin(27770T)>0 270/m 

where 15 is an indicator function that is equal to one when statement S holds and zero otherwise. 
The bounds on and are 


iS77„.7) ,, /47°^ P-77.7)-i?;-(70. r)) 




cos(2777or)<i?j!j'(7o,T) 9 f2 

J mir 


^sin(27r7oT)<0 


7r/cSin(27r7or) 


B\ (70, r) := 


/min 

fc (cos (27r7or) - (70, r)) 


tt/c sin (27r7or) 

-^sin(27r7oT)>0 r •, 

/min 


1 - 1 


TT^r^ 

cos(2777ot)>BJ^(70,t) 2 ^2 

min 


(B.IO) 


(B.ll) 


dU /_i 2/c {Bq ( 70 , r) - cos (27r7or)) 7 ^ 

2 (.70)'Tj • cos(2777ot)>BJ'(70 


2 2 \ 2 
TT^r^ ^ 


^)2f2. 

mm 


B2 (70, t) := 


- 471^70/0-^0 (70, t ) + l/ 7 U(^g^^)>o/c (70, t ) , 

‘^fc (-^0 (70, t ) - COS ( 27 r 7 oT)) 


1 - 1 


2 2 \ 2 


cos(2777or)<Bj'(7o,T) 9 ^2 


2p. 

mir 


where 


(70, t) = 


h\ (70,7-) = 


47r^7o/c( 70 , r) + ^2 (7o, r 

471^70 


(B.12) 


(B.13) 


/min 

471^70 

/min 


1 - 1 


sin(27r70T)<0 9 r2 

J min 


TT^r^ ^ sin ( 27 r 7 oT) 


2-7170 r 


--Bo(7o,t) , 


1 lsin(2-7r7oT)>0 9 ^2 

■^/ mi 


Ti^r^ \ sin ( 27 i 7 or) u 


27170 r 


-Bq ( 70 , r) , 
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Bs ( 70 , r) := ( 70 , r) (^1 + (^ 

, 2 /c/i^( 7 o,r) 


7o/min ‘^lofmm 


1 - 1 




^3 (T' 0 ,t )<0 2 ^2 

J mir 


(B.14) 


^3 ( 70 , r) := - 47 r 27 o/c 5 }^ ( 70 , r) 


1 + l_BU(7o,r)>0 


1 


1 


7o/min 27g/j 


2 f2 
min 


‘^fch^ ilo,T) / _ ^ 




+ I ^h^(lO,T)>0 2f2 j ’ 

J min / 

where the definition of ( 70 , t) and /13 ( 70 , t) is separated into two intervals; 


(B.15) 


, C/ . . fhg,r, 3 ,a (70, r) 4 „^_, 3 _„( 7 o,r )>0 " ^Lr,3,6 (^0,^) if 0 < |t| < Tg (70) , 

I7„,,(70.r) it M >.3(70). 

_ lhLr, 3 ,a ( 70 ,r) „( 7 o,r )<0 " ^Lr,3,6 (^ 0 ,^) if 0 < |t| < Tg (70) , 

^ ’ l^£r, 3 ( 7 o,'r) if |r| > r 3 ( 7 o). 

In particular, T 3 (0.247) = 0.4614, T 3 (0.339) = 0.2346 and T 3 (0.414) = 0.3051. Finally, we have 

r23„„ r2 


.u 47^0/? /-_ sm^or) ^ ^ ^ 


^near, 3 ,a (7O)'^) • 


^near, 3 ,a (7O)'^) • 


/min 

47r27o/c /dL 


fn 


B { 70 ,r)- 


2tt7ot 
sin (27r7or) 




sin(27r7or)>0 3^ ^2 j j ’ 

m 




27r7or i ’ 


^near, 3 ,f) ( 70 ) '^) • 


/2 (^31.37r^7^r2 - 167r®7^r^ (l + 


70 fn 


1 - 


2/. 


^— 
min 


15(2 + 7 ^;; 


^near, 3 ,f) ( 70 ) '^) • 


2vrSoVcV(8.17 + 


20 


70/ir 


15 1 - 


6/" 


I U / \ fcBi { 70 , Tj . 

^far ,3 ( 70 , r) :=--- ( 1-1 


(^f iiB}^(70,-r)<0 2j2 ^ -S2(70 ,t), 


^L . /c^i^(70,r, , , ^ 

%ar,3 (70, t) := --- (1-1 


(f ^5^70,r)>0 2^2 ) ^^(70 ,^). 

V min / 


(B. 16 ) 
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e=i 






■ 4')(/er)//f (/, = 103 ) ♦ <)(/,r)//,M/c =5 103 ) X (f^r) / f, {f, = 10 ^) 

— Bl,{T)/fi _ — B\,{T)l!i _ 


Figure 14: Upper and lower bounds on and its derivatives. 
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B.2 Bounds on the tail of the kernel in Lemma 14.61 


The bounds are plotted for 7 = [0.247,0.339, 0.414]^ in Figure 


15 


^ 7,0 ('^) := 

i=i 

p p 

b'yAir) ■='^bi{'yi,T) boi'ij,T), 

i=l 

P P P P P 

b^,2{T):=^b2{-fi,T) n bo{-fj,T) + Yl Y1 bi{ji,T)bi{'yj,T) 6o(7fc,T), 

i=l i=l k=l,k^{ij} 

P P P P P 

b-f,3 (t) ■— 'y ^ ^3 (7i) '^) bo {'jj, r) + 3 E E b 2 {li,T) bi {jj , r) bo (jk , t) 

i=l i=l j=lj^i 


P P 


+E E E bi{ji,r)bi{-fj,T)bi{jk,T) bo{ji,T), 

i=l j=l,j 7 ^i k=l,k^{i,j} 


(B.17) 


where 


bo ( 70 , t) := 


bi (70, t ) := 


27r7or(l-|^ 

\ min 

/c(l + ^0 ( 70 ,'t)) 


b 2 ( 70 , r) := 47r^j^f^bo ( 70 , r) ( 1 + — I —^ 


bs ( 70 , t) := 4T?-iljlbx ( 70 , r) { 1 + 


7o/min 

1 


7o/n 


+ — {b 2 ( 70 , t) + 


fcbi ( 70 , t) 


C Proof of the bounds on the interpolation kernel 

C.l Proof of Lemma 14.41 

We assume that t is between 0 and 0.5. The bounds for t between -0.5 and 0 follow by symmetry 
{K and are even, i^7) and are odd). The following local bounds, which are derived from 
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£=0 


£=1 



■ ■ ■ ■ 

K? ifcT) 
K? ifcT) 

■ 1 

O O 

T—1 T—1 

II II 

K? ifcT) 

b'r/ (t) / 

//f (/o=5103) 

fi 


Figure 15: Non-increasing upper bounds on the absolute value of and its derivatives. 
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Taylor series expansions around the origin, will be used often during the proof. If n > 0 

,3 


u 


u -< sin (u) < u, 

6 




u - 1 -< sin lu) < u - 1 - 1 -, 

6 120 5040 “ ^ 6 120 5040 362880’ 


u 


1 —^ < cos (u) < 1 , 


2 4 

u u u 


2 4 

U U 


1- 1 -< COS iu) <1- 1 - 

2 24 720 “ ^ ^ “ 2 24 


and if 0 < rt < \/2 


u < tan {u) < 


1 _ 

^ 2 


< sin {u) tan (u) < + 

< sin (u) tan^ (u) < + 


2 - 1 / 2 ’ 


u 


(4 - 1/2) 


(C.l) 

(C.2) 

(C.3) 

(C.4) 

(C.5) 

(C. 6 ) 

(C.7) 


(2 -1/2)2 

Let us fix a value 0 < 70 < 1 and set fn = 70 fc to alleviate notation. Henceforth we assume that 

/. > By PH and pa 


1 1 1 
TTt - sin (vrt) - 

V 670/min/ 


1 _ /iS*" 


270^/. 


TTt 


2 

min ^ 


1 


1 

< — . 


tan (vrt) irt 


(C. 8 ) 

(C.9) 


We begin by deriving bounds on K (70 fc,t) near the origin. By (C.2) 


sin (( 2/0 + 1) TTt) 2/0 
2/0 + 1 sin (27r/ot) 

sin (( 2/0 + 1) TTt) 2/0 


< 


1 - 


7r^(2/o + l)^t2 7r^(2/o + l)^H _ w^(2fQ + lft<^ ■K^{2fo + lft^ 

6 _ 120 _ 5040 


362880 


, 27r2/n"i" , 47r6/6/6 

4 Q “T 


15 


315 


> - 


1 - 


7 r 2 ( 2 /o + l)^t2 yr4(2/o + i)4t4 _ yr6(2/o + l)S/6 

6 120 5040 


2 fn + l sin (' 27 rfnt') 1 27 r 2 / 2 t 2 2 -K^f^t^ . 27r8/8/8 

•' V du / i-^-1---1-ga-jc; 


315 


2835 


Combining this with (C. 8 ) establishes 

^near ,0 (TO , fct) < K (70 fc,t) < ^ (73 J^t) . 

To prove 

^far ,0 ( 70 , fc t) < 77(70 fc, t) < B}i^ Q ( 70 , fc t) , 

we express the kernel in the following form 

77 1 f f \ - (27r/ot) cos (iTt) cos (27r/ot) 

( 2 /o + l)sin( 7 rt) + ( 2 /o + 1 ) ’ 


(C.IO) 

(C.ll) 
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and apply (C.3) and (C.9). 

Some basic trigonometric identities yield 
vr 


{h,t) = 


cos((2/o + l)7rt) - 


'cos ((2/o + 1) nt) - K (/o, t) cos (Trt)) 


= TT 


sin (vrt) 
TT 

sin (vrt) 

TT 


sin (Trt) 

'cos( 27 r/ot) - K (/o,t) 


sin ((2/o + 1) Trt) cos (vrt) 
(2/o + l)sin (Trt) 


cos (2Tr/ot) cos (vrt) — sin (27r/ot) sin (rrt) — (/o, t) cos (rrt)) 

- sin (2Tr/ot) ) . 


(C.12) 

(C.13) 


tan (-Trt) 


This implies 


(A,«) > , ( coB(2,/ot)- ByOo./ot) _ 

\ tan (TTcj 


Applying (C.9) then establishes 

Si^(7o,/ci) <i^(')(7o/c,i) <BB['(7o,/ct). (C.14) 

To bound the second derivative of the kernel we again leverage some trigonometric identities, 

(/o, t) = -tt^K (/o, t) ((2/o + 1)2 - l) - (C.15) 


= -4TT^Kifo,t) {f^ + fo)- 


‘Itt^ 


tan (rrt) 


\TTt) 

cos(2Tr/ot) - K (/o,t) 


tan (rrt) 


- sin (27r/ot) 


=-dTrVo'i^ (/oT)- 


2 Tr^ (cos ( 27 r/ot) - .g (/q, t)) 
tan^ (-/rt) 


+ 4Tr^/o 


sin (2Tr/ot) 
2/o tan (vrt) 


-Kifo,t) . 


This implies 


A'® (A.«) < ( 7 „. /.*) + 

A«^) (/,t) > (70./02) + ^’-7g.770.W - oos(2./ot)) 

tan yTTij 


Applying (C.9) yields 


B2 (70, /c 0 < a:^ 2 ) (^0 t) < ^ 


(C.16) 
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Using the fact that the derivative of 1/ tan (u) is —1 — 1/ tan^ (u), we rewrite the third derivative 
of the kernel as 


(/„,*) = (/„,t) ({2/„ + 1)^ - 3) - ^ (/0.t) - (C.17) 

= -2.^Km (/„, t) (2/7 + 2/0 - 1 ) - - 4.A- (/„,t) (/7 + /„) - 

= -2,7aCI (/„, 0 (2/7 + 2 /„ - 1) - ^4/„ (Him - K (A. *)) + 4*)) . 


To bound z{t) we perform some algebraic manipulations and then apply (C.2), (C.4), (C. 6 ) and (C.7), 

4 4-2 A -7 4 - 4 / , sin(27r/ot) 3 (cos (27r/ot) - K (/o, t)) 

.(t) = -4/„ A (/„, t) + -- 

_ (-4/o + 2 /o + 1 ) sin {2-71 fpt) ^ cos (27r/ot) 

(2/o + l)tan( 7 rt) 2/o + 1 

_ 3 ((2/o + 1) cos (27r/ot) sin (/rt) - sin ((2/o + 1) Trt)) 

( 2 /o + 1 ) sin (vrt) tan^ (vrt) 

sin (27r/ot) ((—4 /q + 2/o + l) sin (vrt) tan (/rt) + 3 cos (vrt)) 


+ 


+ 


( 2 /o + 1 ) sin (vrt) tan^ ( 7 rt) 
cos (27r/ot) (—4 :/q sin (vrt) tan^ (vri) — 6 /o sin (vrt)) 

( 2 /o + 1 ) sin (vrt) tan^ (vrt) 

(27r/of - “ ^i) ((“‘^/o + 2/o + 1) (TT^i^ + £ 2 ) + 3 (^1 - ^ + 63 ^^ 

( 2/0 + 1 ) sin (vrt) tan^ (nt) 

(1 - 27r2/2t2 g (^- 4 / 0 ^ (vr^t^ + g/g, 


( 2/0 + 1) sin (vrt) tan^ (vrt) 

(sT ^ ^ + 2o(/o,i, £1,62,63, €4,65, ee)) -871^/0^^ ^ 

15 ( 2/0 + 1) sin (vrt) tan^ (-irt) 


where 


315 


kil < 


|.4| < 


| 62 | < 


ksl < 47r'’t'’, 


|63|< 

| 66 |< 


24 ’ 


45 , , o, _ , ' - 120 

If /c^ < k and /min > 10 ^ some straightforward, yet tedious, computations yield 


ko (/0,T6l,62,e3,e4,65,e6)| < 0.17, 
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which together with (C.8) and (C.9[) implie^ 


(si.SvrVo^t' - levrVo®^^ (l + 


70 /it 


1 _ 

2^o7^ 


15 2 + 


70 /it 


< z (t) < 


27r2/o/^ (8.17 + 


20 


70 /it 


15 1 


67g/ln 


This establishes 


^3 ( 70 , fct) < fc, t) < (70, fct) 


(C.18) 


for 0 < /ct < Ts (70). For fct>T^ (70) the bounds are obtained by combining (C.17) with (C.9). 


The bounds (C.IO), (C.ll), (C.14), (C.16) and (C.18) combined with the identities 


{t) := ^ (7,/,, t)llK (7,/,, t), 


i=l 

P 




(i) - ^ ( 7 ,/,, t) K (7,/e, t) + (7jc, /) (7,/e, t) n ^ ’ 

2=1 J+2 2 = 1 J+2 kf{i,j} 

«) - E (life, t)llK (7,/e, t) + 3 iF(2) (^./^, t) ^ kW (^ ./^, f) n ^ 

J+2 2=1 jy2 


i=l 


+ K^^Hlkfct) n (C.19) 

fc^{2,i} ^^{2,/+} 


2 = 1 


yield 


B^^,{f,t)<K!f\t)<B};^eifct). (C.20) 

Finally, we control the deviation of the kernel and its derivatives from their values on a grid. 
Bernstein’s inequality allows to bound the /th derivative of a trigonometric polynomial. 

Theorem C.l (Theorem 4 [62]). Let P he a trigonometric polynomial of order N 

N 

^ (^) = X] ’ (C-21) 

i=i 

such that \P {0)\ < 1 for all real 9. Then, 


pW (0) 


< iV^ 


for all real 9. 


®In this step we assume that 70 > ll 32 fc, which holds for the values of 70 we are interested in. Otherwise we 
would have to add a term — 5 / 8 /o in the hrst factor of the numerator of the lower bound. For the upper bound, we 
assume 70 > 1 / 2 // Similarly, this holds for the values of 70 that we consider, but we could add an extra term to the 
numerator of the right hand side if this were not the case. 
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Applying the theorem to K^{t), which can be written as P (27rt) for P of the form (C.21) for N = fc 
and has magnitude bounded by 1, we obtain 

< ( 2 ,r)Vc'- 

As a result, for any t such that |t — < e/fc 

< (27r)^+^ f^e 

by the mean-value theorem. Combining this with (|C.20 ) completes the proof. 


C.2 Proof of Lemma 14.61 


By (C. 8 ) 


< 


|sin((2/o -b l) 7 rt)| 
(2/o + 1) |sin ( 7 rt)| 
1 


(C.22) 




As long ast < this bound is decreasing because the derivative of the denominator is positive. 

By (C.12) and (C. 8 ), we have 




< 


< 


7 r(|cos(( 2 /o + l) 7 rt)| \K (/o,t)| |cos ( 7 rt)|) 


I sin (7rt)| 


(C.23) 


1 + ^0 (70, fct) 




^ ^ 67o/min 


The bound is again decreasing if t < because the derivative of the denominator is positive 

and bo is decreasing. By (C.15) and (C.9) 

(/o, t)\<7r^\K (/o, t)\ (( 2 /o + 1 )' - l) + (C.24) 

( 70 ,/ct) 


< 47r /o^o ho, fct) 1 + 


1 

70/n 


+ 


If f < the bound is decreasing because so are bo and bi. The same holds for the next bound, 

which results from combining (C.17) and (C.9), 


K^^Hfo,t) <7r^ 


{fo,t)\ (( 2 / 0 + 1 )' 


-3 + 


27r 


tan (vrt) 


(/o,t) 


+ 


n\KW (/o,t)|' 


|tan (7rf)| 


(C.25) 


< dvr fobi ho, fct) 1 + 


70/n 


+ T ^2 ( 70 , fct) + 


h ho, fct) 


These inequalities together with (C.19) establish the bounds in the lemma, which are decreasing 
in T if T < 450 < (which implies t < if /min > 10^) because they are products of 

positive decreasing functions. 
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C.3 Proof of Lemma 14.71 


We bound the first term of the decomposition 


E 




= E 

t,ern(o,|; 


K? (t-t,) 


+ 


E (* - *i) 


tj&T\{o} tjern(o,|) tjern[-|,o) 

the argument for the second term is almost identical. 

Without loss of generality we assume that the elements of T n (O, are ordered so that 0 < ti < 


t 2 <■■■< 1/2. By Corollary 4.5 


for any u such that fc (tj — t) — e < u < fc {tj — t) + e. Maximizing this quantity on an equispaced 
grid Qj^r with separation e that covers the interval [jTmin — t , (j + 4) Tmin] for r — e < /ct < r yields 
a bound that is valid for all values of tj such that tj — t < (j + 4) Amin, since tj — t > jAmin — t 


due to the minimum-separation condition. If tj — t > {j -|- 4) Amin, then by Lemma 4.6 

K/p {t - tj) < {{j + 4) Tmin) • 

Taking the maximum of the two bounds allows to control the contribution of the kernel (or the ^th 
derivative of the kernel) centered at tj. We take advantage of this for 1 < j < 20 and then apply 
the coarser bound 

1/2) Tmin) 

for 21 < j < 400. This bound follows from Lemma |4.6| and the fact that tj — t > {j — 1/2) Amin 
by the minimum-separation condition. The remaining terms can be controlled using the following 
lemma, which we prove in Section [C.4[ 

Lemma C.2. For fc > 10^, i E {0,1, 2,3}, 7 = [0.247, 0.339,0.414]^ and 80/fc < |t| < 1/2 

Af>(t) <67{t):=gT'' " ' 


nti7. Wet 


1.1 


By the lemma, 

E 

tjeTn(400An 


KpHt-tj) < ^ 


A:=400 


400 


= Kt (^^min) - X] (^"^min) 

k=l 

(2vr/c)' 


k=l 


nLiT V4 t. 


1.1 




1 


/cP 

\k=l 


400 

E 

k=l 




(C.26) 


The infinite series k ^ is equal to the Riemann zeta function (/ (p), so we can easily compute 

Cl for the desired value of 7 and conclude that 

\KlfHt-t,)\<H{T). 

t,ern(o,i) 
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C.4 Proof of Lemma 1C.21 


In this proof p := 3 (the lemma also holds if we convolve a different number of Dirichlet kernels). 
To obtain bounds that are valid for large t we use a simple lower bound on the sine function that 
holds between 0 and | 


2t < sin (irt). 

Combining it with (C.8) and the bounds (C.22), ( C.23| ), (C.24) and (C.25) on the magnitude of 
the kernel and its derivatives from Section [C. 2 yields 


\K{fo,t)\ < 


1 1 

< 


(2/o + l)2t - 4/of’ 

TT / 1 

<w: 1 + 


2t V 4/of 


< 


vrVo 


2vr3/2 


1 + 


7o/n 


vr 


+ VT 1 + 


1 


< 


TT^/O 


1 + 


1 


+ 


4/ofo 

1 


f2 \ 4/ofy f 

1 \ 1 
1 + ^^ + 


7o/n 


1 + 


1 + 


1 ^ + m 


70/n 


+ 


TT/of 


1 ^ + m 


7 r/of \ 70 /i 


+ 


7r/of 


1 + 


2vr2/2t2 4fQt^ 

If 7 o ^ 0.25 then l//of <0.05 for f > ^, which allows to coarsely summarize these inequalities as 


1.12^-V/o"-i 


for j G 0,1, 2, 3 and /min > 10^. Together with (C.19) this implies 

\KAt)\< 


1.1 VA 1 


n 


4'^ it) 


4/cf; ^^7*’ 

<27r/, 


^ ^ \P ^ ^ 1 

4 ^ n 


42 ) (t) < ( 2 vr/,)' 


43 ) (f) < (271/,)' 


1.1 


p p 


p 

p 

+ E 


1.1 


E 

i=l 
P P 

E 


7i 


7f 


37i 


4 f f / . . O' • ' ^ E n 

' i=l \ ” j=l ■*■^=1:^7^®li k=l,kj^i,j m=l,mj^i,j,k 


1 


,• 7. 7m 


Since by definition 7j = 1) following identities complete the proof, 


E n y = ETy^ = n4 

i=i j=i4^i i=i 1 li=i 1 li=i O'j 
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E 


7i 


= 1 \ nj = l 7i 


+ E n 

j=l,j^ik=l,k^i,j 


Ik 






2=1 

P 




nj.iT/ 


S (rF^^+ S n 


1 

7m 


E 


7* 


+ E 


27i + 7i 


+ E n 


=1 ynj—lj^i7j Y0fc_l,fc^ij7fe j 7m ^ 

E f+ E f27*"7i + la] + E 

ij—1 •? i=l Y j=lj^i Y k=l,k^i,j 




E 7*^ + ( 7 I 


E I = 

j=ijy* 


n^.i7,- 


C.5 Proof of Lemma 14.91 

By the fundamental theorem of calculus, if t' is such that je <t'< {j + 1) e for some integer j 

ft' 

F' {t') = F' (0) + / F” {u) dn 

Jo 

^ HZ+l)e 


1=0 


r 

F"{u)du+ / F"(M)du. 

Jf 


This implies 


7-1 


F ' { t ') < e ^ F2 ,z + 1f2,j>oP2j 


(C.27) 


d=o 


Now, recall that ke < t < {k + 1) e. By the fundamental theorem of calculus, the assumptions on 


F and (C.27), 


F{t) = F (0) + F' (u) du 


PJ ft 

/ F'{u)du+ / F'{u)du 
A-n J F J ke 


= i + E/ 

i=0 

/k-1 /i-1 

1 + I E ( E'^^,/ + 1 f2,j>oP2j I + 1 f2>oP2 

\j=0 \i=i J ) 


\ 
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